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(54) Variable resolution view-tracking display 

(57) A panoramic display apparatus 1 0 mounted, at least in part on a helmet 14 to be worn by an operator, in which 
apparatus there is a projection means 16 for projecting a high resolution stereo display within a small first field of view 
containing the foveal line of sight of an operator's eyes and a low resolution display outside said first field of view. The high 
resolution first field is used to present a detailed image of the object the operator is looking directly at whilst the low 
resolution field presents a taw resolution peripheral image in the areas where the operator's eyes are incapable of 
perceiving high resolution images. One or other or both displays may be projected onto an eyeshield 24 or displayed via 
one or more active matrix liquid crystal displays which may be mounted on or form part of the eyeshield 24. 

The apparatus may be used with flight or other simulators, remote control apparatus e.g. robots, surface or space 
vehicles. Alternatively the apparatus may be combined with cameras to give an aircraft pilot an "all-round" view (fig 6). 




At least one drawing originally filed was informal and the print reproduced here is taken from a later filed formal copy. 
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AN EYE- SLAVED PANORAMIC DISPLAY APPARATUS 

The present invention relates to an eye-slaved panoramic 
display system and relates particularly, but not exclusively, to 
such an apparatus for use in applications that require remote or 
synthetic visual contact with the environment such as vehicle 
control, remote piloting, telerobotics and simulation. 

Presently known systems for outside-world representation 
generally employ one of three methods for image display, namely: 
direct view monitor, video projection or, head/helmet mounted 
display (HMD). Direct view monitors, such as cathode ray tubes 
(CRT's), offer reasonably high image resolution and are a 
popular display technique, both for sensor-derived and 
computer-generated images but they offer only a comparatively 
narrow field of view. The field of view can be increased by 
increasing the screen size but this cannot be done above a 
certain diame ter without reducing image resolution or refresh 
rate. This can also be prohibitively expensive and can incur 
unacceptable size and weight penalties. 

Video projection systems that employ multi-screen or 
dome-wall video projections can provide a large instantaneous 
field of view which in many cases is truly panoramic with over 
120° of view. Such systems provide the viewer with valuable 
peripheral information as well as a more realistic viewing area 
and, consequently, have become very popular in the field of 
flight simulation. These systems suffer from the problept of 
reduced image quality because the gain in field of »ew is* 
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obtained by simply expanding the original image across a larger 
surface at the expense of image quality (resolution and 
contrast) - 

The ESPRIT projector-based systems proposed by 
Singer-Link provides a so-called area-of -interest (AOI) visual 
simulator. In this system a wide-angle low resolution image is 
projected onto a simulator dome and is overlaid with a higher 
resolution, but smaller, eye-slaved image, provided by a 
separate servo-driven projector* Such a system does not lend 
itself to stereoscopic image presentation, nor to use by multiple 
viewers . 

Head or helmet mounted display systems are being 
developed which present a condensed CRT-type image onto a 
head or helmet mounted eye piece. These can offer high 
resolution and with appropriate viewing conditions, good 
luminance contrast. The images are normally head-slaved (and 
could also, in theory, be eye-slaved) so that the wearer is able 
to guide his/her direction of gaze through a total potential field 
of view of up to 360°. In addition HMD systems can also 
provide binocular stereopsis , something which can only be 
approximated with direct-view and projection displays. 
Unfortunately the field of view provided by HMD systems is 
relatively small (eg 60° binocular FOV, including 40° per 
eyepiece) and increasing the field of view without loss to image 
quality is prohibitively expensive in terms of ima g e 
generation/conversion. 
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None of the above mentioned systems is capable of 
providing a suitably sharp stereoscopic image with a large 
general viewing area without impairment of image quality or 
without overstretching the image generation /conversion systems. 

It is therefore an object of the present invention to provide 
a panoramic display apparatus which mitigates the problems 
associated with the above mentioned systems. 

Accordingly the present invention provides a panoramic 
display apparatus comprising a head position tracking device and 
an eye position tracking device which in combination enable an 
operator's foveal line of sight to be determined, means for 
dividing an operator's field of view into a first field containing 
an operator's foveal line of sight and a second field outside said 
first field, and projection means for projecting a high resolution 
display within said first field and a low resolution display in said 
second field. 

It will be appreciated that the above mentioned apparatus 
exploits the limitations of the human eye, which is incapable of 
perceiving high resolution images outside a narrow 
(approximately 5°) cone around the foveal line of sight, and 
provides a panoramic display apparatus that gives high image 
resolution only where the operator's eye is capable of perceiving 
it and a low resolution display outside this field. Such an 
apparatus is capable of providing a suitably sharp image with a 
large viewing area without impairing image quality and without 
over-stretching the image generation/ conversion system. 

Preferably, the apparatus includes an eyeshield of an 
operator's helmet onto which the high resolution display is 
projectable and a remote surface onto which the low resolution 
display is projectable. 



Alternatively, the apparatus includes an eyeshield of an 
operator's helmet onto which the high and/or low resolution 
displays are projectable. 

Conveniently, the above mentioned remote surface may be 
the interior surface of an aircraft canopy (or some other 
workspace interior). 

The projection means may comprise a first projection system 
for projecting the high resolution display and a second projection 
system for projecting the low resolution display. In a 
particularly advantageous arrangement the high resolution 
display may be projected onto the eyeshield of an operator's 
helmet and the low resolution display is projected onto a remote 
surface such as the interior surface of an aircraft canopy. 

In a simple arrangement the first projection system may 
comprise a first projector which presents an image alternately to 
both left and right eyes of an operator thereby to give a stereo 
image. 

If just one projector is used in this way, the projection 
system may include an LCD shutter located at or near a source 
of said image and passive polarisers before an operator's eyes in 
order to present the image alternately to both left and right 
eyes. 

Alternatively, the first projection system comprises first 
and second projectors locatable and operable in combination to 
present an image to both eyes of an operator in a manner which 
gives a stereo image. 
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One or more active matrix LCD arrays may also be used to 
project the high and /or low resolution displays. The array 
is /are transparent and locatable to form part or all of an 
eyeshield of an operator's helmet or is/are mountable on said 
eyeshield. 

Conveniently the first field of view may extend up to or 
slightly beyond an operator's foveal boundary. Gradation means 
may be provided for modifying the resolution of the first field at 
its periphery in a manner which matches or approximates the 
reduction in image resolution due to the human eye's 
acuity/ eccentricity curve. 

Computation means may be provided for computing the 
visual field location or locations of the high resolution first field 
or fields. 

Conveniently, the first projection system may be mounted 
on a helmet to be worm by an operator and the second projection 
apparatus may be mounted remote from said helmet, thereby to 
enable the low resolution second field projection to be projected 
onto a remote screen. 

In a particularly advantageous arrangement the second field 
of view extends to form a full field of view of up to 360° about 
an operator's X and Y axis, thereby to allow the operator to 
see in all directions including downwards and rearwards. 

The above mentioned panoramic display apparatus may be 
used remotely to pilot a vehicle and/or in a robot/robotic 
system, thus avoiding exposing the operator to dangerous or 
inhospitable environments. 
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Advantageously two or more panoramic display apparatus 
may be linked to each other to form a network of visual displays 
for use with multiple operators and different information may be 
presented to each operator's high resolution first field of view 
thereby to provide the operators with only that information they 
require in order to perform their given task. 

In a particularly convenient arrangement a helmet may be 
provided with a panoramic display apparatus as described above* 

The present invention will now be more particularly 
described by way of example only with reference to the 
accompanying drawings in which: 

Figure 1 is a cross sectioned side elevation of an operator's 
helmet incorporating a panoramic display apparatus according to 
the present invention and an aircraft canopy, 

Figures 2 to 4 illustrate various alternative projector 
positions. 

Figure 5 is a front elevation of two operators helmets 
incorporating panoramic display apparatus according to the 
present invention together with associated computation means, 
and 

Fig 6 is a perspective view of an aircraft incorporating a 
number of features the subject of the present invention. 

Referring to Figure 1, a panoramic display apparatus 10 
according to the present invention comprises a head position 
tracking device, such as an accelerometer 12, mounted on an 
operator's helmet 14, an eye position tracking device in the form 
of, for example, an infra-red LED oculometer 15 and a projection 
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means shown at IB. In a preferred embodiment the projection 
means comprises a first projection system 18 for projecting a 
high resolution stereo display within a first field of view 
containing the foveal line of sight of an operator's eye 20 and a 
second projection system 22 for projecting a low resolution 
display outside said first field. It will however be appreciated 
that only one projection system 18 may be used if suitably 
modified to be able to project both the high and low resolution 
displays. 

The first projection system 18 is preferably mounted on the 
operator's helmet 14 and arranged to project the high resolution 
display onto an eyeshield 24 of the helmet 14. The second 
projection system 22 may be arranged in any one of a number of 
ways as illustrated in figures 2 to 4 inclusive. In figure 2, the 
second projection system 22 comprises a number of static 
projectors 22a, b projecting onto, for example, the inner surface 
of a flight simulator dome. Alternatively, just one or two 
wide-angle, multi-axis, servo-drives, projectors may be provided 
and mounted, for example, above the operators head as shown in 
figure 3. Figure 4 illustrates yet another arrangement in which 
static video or laser projector(s) slaved to head-position by 
means of a flexible fibre optic link to head-position by means of 
a flexible fibre optic link 13 to the helmet; the images would 
then pass through a wide-angle projection device 22c mounted on 
the front of the helmet 14 before being projected on the remote 
surface 15. In a particularly advanced and compact 
arrangement, the second projection means may be arranged to 



project the low resolution display onto the eyeshield 24 of the 
helmet 14 alongside the high resolution display (see for example 
figure 1). 

The first projection system 18 may include one or two 
projectors 18a, b. When just one projector 18b is used the 
system is further provided with a Liquid Crystal Shutter 26 at 
the image source and passive polarisers 28 located before the 
operator's eyes 20. In operation the shutter acts to deflect the 
source image alternatively to operator's left and right eyes in a 
manner which, in combination with the passive polarisers 28, 
acts to present an operator with a stereo high resolution image. 
When the two projectors 18 are used each may be arranged to 
project the high resolution image to one or other of an operators 
eyes thereby to provide a stereo image. 

In a still further alternative arrangement the first 
projection system 18 is formed by one or more active matrix 
liquid crystal displays 30, 32 well known in the art and 
therefore not described herein. With suitable image processing 
just one display may be used to present both the high and the 
low resolution fields and the said one display is preferably 
mounted on or forms part of the eyeshield 24. When two 
displays 30, 32 are provided it will be appreciated that only that 
display presenting the high resolution display need be provided 
on the eyeshield. The display presenting the low resolution 
display can be provided on or form part of the eyeshield 24 or 
can be positioned remote from said eyeshield 24 if desired. 
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When the one or more displays 30, 32 are mounted on, or 
form part of the eyeshield 24 it or they may be of a transparent 
type thus allowing the operator to see real images which are 
present beyond the eyeshield 24, such as, for example, aircraft 
cockpit instruments and the like or a remotely projected low 
resolution display. 

The oculometer 15, which monitors the eye position, and the 
head position sensor 12 are each linked to a computation device 
34 via cables 36, 38 respectively- The computation device is 
configured to monitor the outputs from the oculometer and 
position sensor and determine the direction of the operator's 
foveal line of sight such that the high and low definition images 
may be projected relative thereto. Such an arrangement is well 
known and hence forms no part of the present invention other 
than to the extent that it enables the present invention to be 
operated. 

In order to exploit fully the limitations of the human eye, 
which is incapable of perceiving high resolution images outside a 
narrow (approximately 5°) cone around the foveal line of sight, 
the computation device 34 is also configured to determine the 
boundary of an operator's high resolution image perception and 
define the boundary of the display apparatus high resolution 
first field relative thereto. Preferably, the boundary of the 
first field extends up to or slightly beyond an operator's foveal 
boundary thereby to minimise the size of the high resolution 
display and maximise the size of the low resolution display and 
hence reduce to a minimum the computation power required to 
present the overall image to the operator. It will be appreciated 



that the operator's head and eyes will be constantly moving as 
he observes the display presented before him, consequently the 
computation means will be constantly updating the position of the 
first and second fields in order to provide the best possible 
display to the operator. Reduction in the computation power 
required to provide the overall display frees computation power 
to be used to speed up the image update rate thereby providing 
a more realistic overall image. 

In operation, the high resolution first field may be used to 
present vital information to the operator, such as weapon or 
aircraft performance information, together with an enhanced view 
of that portion of the overall display the operator is looking 
directly at. Portions of the low resolution display may be 
enhanced in order to attract the operator's eyes to potential 
targets if desired. 

Figure 6 illustrates an aircraft 40 which is provided with a 
number of remote cameras 42, 44, 46, 48 and 50 respectively set 
for forward, downward, upward and left and right viewing 
respectively. These cameras may be linked into the panoramic 
display apparatus 10 to provide the pilot operator with outside 
world visual information in all directions, including directions he 
would normally be unable to see. It will be appreciated that a 
pilot operator of an aircraft equipped with such cameras and 
display apparatus will be able to look down by rotating his head 
about axis Y-Y towards the cockpit floor for example and see a 
representation of the real world image on the other side thereof 
thus enabling him to make visual contact with landmarks or 
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aircraft normally outside his field of view. If the pilot operator 
rotates his head about axis X-X he will be able to make visual 
contact with landmarks etc to his left and his right as well as 
behind the aircraft if a suitable remote camera if provided to 
point in this direction. The computation means may be 
configured to be selective with the outside world information it 
presents to the pilot operator, thus preventing him from being 
distracted by irrelevant detail. 

An aircraft when provided with the above mentioned 
cameras and suitably modified to enable remote operation of the 
aircraft controls to be achieved lends itself to being remotely 
piloted by a pilot operator using a panoramic display apparatus 
10 as described above. The operator would be provided with all 
the visual information he requires in order to fly the aircraft 40 
without being exposed to the often hostile environment the 
aircraft is flown in. 

It will be appreciated that whilst the panoramic display 
apparatus 10 has been described with reference to use on 
aircraft it also lends itself to use with manned/ unmanned surface 
vehicles and submersibles, robotic systems in nuclear power 
plants and teleoperation in space. 
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CLAIMS 



1. A panoramic display apparatus comprising a head position 
tracking device and an eye position tracking device which in 
combination enable an operator's fovea! line of sight to be 
determined, means for dividing an operator's field of view into a 
first field containing an operator's foveal line of sight and a 
second field outside said first field, and projection means for 
projecting a high resolution display within said first field and a 
low-resolution display in said second field. 

2. A panoramic display apparatus as claimed in claim 1 
including an eyeshield of an operator's helmet onto which the 
high resolution display is projectable and a remote surface onto 
which the low resolution display is projectable. 

3. A panoramic display apparatus as claimed in claim 1 
including an eyeshield of an operator's helmet onto which the 
high and/or low resolution displays are projectable. 

4. A panoramic display apparatus as claimed in claim 3 in 
which the remote surface is an interior surface of an aircraft 
canopy. 

5. A panoramic display apparatus as claimed in claim 1 or claim 
2 in which the projection means comprises a first projection 
system for projecting said high resolution display and a second 
projection system for projecting said low resolution display. 

6. A panoramic display apparatus as claimed in claim 5 in 
which the first projection system comprises a first projector 
which presents an image alternately to the left and the right eye 
of an operator thereby to give a stereo image. 
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7. A panoramic display apparatus as claimed in claim 6 in 
which the projection system includes a LCD shutter located at or 
near a source of said image and passive polarisers before an 
operator's eyes. 

8. A panoramic display apparatus as cl ai me d in claim 5 in 
which the first projection system comprises a first and second 
projectors locatable and operable in combination to present an 
image to both eyes of an operator thereby to give a stereo 
image. 

9. A panoramic display apparatus as claimed in claim 1 
including one or more active matrix LCD arrays for projecting 
the high and /or low resolution displays. 

10. A panoramic display apparatus as claimed in claim 9 in 
which the one or more active matrix LCD arrays is /are of a 
transparent type and locatable to form part or all of an eyeshield 
of an operators helmet or is/are mountable on said eyeshield. 

11. A panoramic display apparatus as claimed in any one of the 
preceding claims in which the first field of view extends up to 
or slightly beyond an operator's foveal boundary. 

12. A panoramic display apparatus as claimed in any one of the 
preceding claims including graduation means for modifying the 
resolution of said first field at its periphery in a manner which 
matches the reduction in image resolution due to the human eye's 
acuity/eccentricity curve. 

13. A panoramic display apparatus as claimed in any one of the 
preceding claims including a computation means for computing the 
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visual field location or locations of the high resolution first field 
or fields. 

14. A panoramic display apparatus as claimed in any one of 
claims 3 to 11 in which the first projection system is mounted on 
a helmet to he worn by an operator and the second projection 
apparatus is mounted remote from said helmet* 

15 . A panoramic display apparatus as claimed in any one of the 
preceding claims in which the second field of view extends to 
form a full field of view of up to 360° about an operator's 
vertical and/ or horizontal axis. 

16. A panoramic display apparatus as claimed in any one of the 
above claims when used remotely to pilot a vehicle and/or in a 
robot /robotic system. 

17. Two or more panoramic display apparatus as claimed in any 
one of the previous claims when linked to each other to form a 
network of visual displays for use with multiple operators. 

18. Two or more panoramic display apparatus as claimed in claim 
17 operable such that different information is presented in each 
operator's high resolution first field of view. 

19. A panoramic display apparatus substantially as herebefore 
described with reference to and as illustrated in Figures 1 to 3 
of the accompanying drawings. 

20. A helmet when provided with a panoramic display apparatus 
as claimed in any one of claims 1 to 18. 
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(54) VIDEO DISPLAY DEVICE 

(57) Abstract: 

PURPOSE: To obtain a compact video display device 
constituted so that a video is prevented from being 
doubly viewed, the observation of the video is not 
obstructed by reflected light, the irregularity of the 
brightness of a pupil is eliminated and the occurrence 
of aberration is reduced while the diameter of the exit 
pupil is substantially enlarged by using two sheets of 
diffraction gratings. 

CONSTITUTION: This video display device is constituted 
so that the diameter of the exit pupil is substantially 
enlarged by arranging a video display element 2, an 
eyepiece optical system 23, the first diffraction 
grating 11 and the second diffraction grating 12 on an 
optical path in order and making the plural exit pupils 
by deviating them by the diffracting action of the 
gratings 11 and 12. A diopter correction lens 24 is 
arranged on the exit side of the second grating 12 so as 
to easily focus the eye 5 of an observer on an object by 
the correction lens 24. Besides, the observed video is 
prevented from being doubly viewed caused by the plural 
pupils. Moreover, the light is prevented from being 
reflected and the irregularity of the brightness of the 



pupil is prevented from occurring. 
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(54) IMAGE PICKUP DEVICE 

(57) Abstract: 

PROBLEM TO BE SOLVED: To reduce fatigue to 
user of a head-mount display by making more 
speedy and comfortable the change in an image 
pickup direction concerning a head-mount display 
image pickup device. 

SOLUTION: A head-mount display 100 is composed 
of a CCD camera 3 (image pickup means) for 
picking up the image in a visual range 2 of a user 1 
mounting the display, a display device 5 (image 
display means) for forming the image picked up by 
this CCD camera 3 on the retina 4a of the user 1 of 
the display, a detecting means 6 for detecting a 
glance direction 2 of the user 1 of the display, and 
a drive means 7 for making the image pickup 
direction of the CCD camera 3 follow the glance 
direction 2 detected by the detecting means 6. 
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